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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 
All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not Included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1308. 

1 . £3 This communication is responsive to 1 1/20/2006 . 

2. ^ The allowed claim(s) is/are 22-43 . 

3. □ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a) □ All b) □ Some* c) □ None of the: 

1 . □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). t 
* Certified copies not received: 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

4. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

5. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1) □ hereto or 2) □ to Paper No./Mail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper No./Mail Date . 

Identifying indicia such as the application number (see 37 CFR 1.84(c)) should be written on the drawings in the front (not the back) of 
each sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d). 

6. □ DEPOSIT OF and/or INFORMATION about the deposit of BIOLOGICAL MATERIAL must be submitted. Note the 

attached Examiner's comment regarding REQUIREMENT FOR THE DEPOSIT OF BIOLOGICAL MATERIAL. 
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EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Namsoon Jung on 3/7/2007. 



The application has been amended as follows: 



22. (Amended) A method for enhancing facial images of a person in a video by 
superimposing virtual object images onto said facial images automatically and 
dynamically in real-time, comprising the following steps of: 

(a) capturing a plurality of images for said person with a sing le or p l ura l ity of at least 
one means for capturing images, 

(b) processing said plurality of images in order to detect the face of said person in said 
plurality of images for an initial face detection, 

(c) tracking the detected face in real-time using a face tracking window , 

(d) verifying said face within the boundary of the face tracking window using a second 
local face detection, 

(e) calculating facial feature coordinate information from said face by applying a plurality 
of facial feature detection approaches and applying a block-processing, 

(f) processing a touch-free interaction by said person to select virtual object images, and 

(g) superimposing said virtual object images onto said facial images automatically and 
dynamically in real-time, 

wh e r ei n ex e mp l ary fac i a l imag e s compr i se ey e s, nos e , and mouth on th e fac e , 
wherein e x e mp l ary the virtual object images comprise a pair of sung l ass e s i mag e , hat 
imag e , or any othor predefined virtual objects wh i ch that can be attached to and 
enhance the facial images, 

whereby the verification of said face within the boundary of the face tracking window 
makes more accurate facial images to be passed on to the next step while enabling 
real-time tracking of the face, and 
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whereby said touch-free interaction enables said person to choose said virtual object 
images on a means for displaying output to superimpose onto said facial images without 
touching any device. 



33. (Amended) An apparatus for enhancing facial images of a person in a video by 
superimposing virtual object images onto said facial images automatically and 
dynamically in real-time, comprising the following steps of: 

(a) capturing a plurality of images for said person with a s i ng le or plura li ty o f at least 
one means for capturing images, 

(b) processing said plurality of images in order to detect the face of said person in said 
plurality of images for an initial face detection, 

(c) tracking the detected face in real-time using a face tracking window , 

(d) verifying said face within the boundary of the face tracking window using a second 
local face detection, 

(e) calculating facial feature coordinate information from said face by applying a plurality 
of facial feature detection approaches and applying a block-processing, 

(f) processing a touch-free interaction by said person to select virtual object images, and 

(g) superimposing said virtual object images onto said facial images automatically and 
dynamically in real-time, 

wh e r ei n e x e mplary fac i a l i mag e s oompris e e y e s, nos e , and mouth on th e fac e , 
wherein e x e mplary the virtual object images comprise a pair of sung l asses imago, hat 
i mag e , or any oth e r predefined virtual objects wh i ch that can be attached to and 
enhance the facial images, 

whereby the verification of said face within the boundary of the face tracking window 
makes more accurate facial images to be passed on to the next step while enabling 
real-time tracking of the face, and 

whereby said touch-free interaction enables said person to choose said virtual object 
images on a means for displaying output to superimpose onto said facial images without 
touching any device. 



The following is an examiner's statement of reasons for allowance: 

The above claims as amended disclose a method for superimposing virtual 

objects on a face by using a face tracking window. The prior art does not disclose the 

specifics of the independent claims. 
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The above features, as explicitly recited, and in combination with the other 
elements of the claim are neither disclosed nor suggested by the nearest prior art of the 
record. 

Prior arts Gao and Soatto, Black, Tamura, Fukuma, and Cohen et al. (7184602) 
do not disclose performing the steps of independent claim 1 when taken as a whole and 
in the order presented. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Hadi Akhavannik whose telephone number is 571-272- 
8622. The examiner can normally be reached on 10:30-7:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on (571)272-7695. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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